video Pertaining to the portion of recorded informa- 
tion that can be seen. 

"* video black In video presentations, the absence of 
pictures and sound, usually at the beginning and end 
of a program and between segments. 

video board Synonym for graphics adapter. 

videocassette recorder (VCR) A device for recording 
or playing back videocassettes. 

video clip A section of filmed or videotaped material. 

videoconferencing Teleconferencing that provides 
transmission of still or moving images, in addition to 
voice, text, and graphics. (T) See also conference 
call, teleconferencing. 

video digitizer Any system for converting analog 
video material to digital representation. 

videodisc A disc on which, programs have been 
recorded for playback on a computer or a television 
set; a recording on a videodisc. The most common 
format in the United States and Japan is an NTSC 
signal recorded on the optical reflective format. 

videodisc player In multimedia, a device that pro- 
vides video playback for prerecorded videodiscs. 

video display terminal (VDT) (1) A user terminal 
with a display screen, and usually equipped with an 
input device such as a keyboard." Synonymous with 
visual display unit (VDU). (T) (2) In SAA usage, 
deprecated term for display device. 

video display unit (VDU) (1) Synonym for visual 
disp ay unit. (2) In SAA usage, deprecated term for 
display device. 

video encoder A device that transforms a high- 
resolution digital image from a computer into a 
standard television signal, thereby allowing the com- 
puter to create graphics for use in video production. 

video gain The strength of a video signal. 

video graphics adapter (VGA) A computer adapter 
that provides high-resolution graphics and a, total of 
-56 colors. See enhanced graphics adapter (EGA). 

video head The mechanism inside a videotape player 
that reads the video information recorded on the tape. 

video lookup table (VLT) A color map implemented 
in hardware. 

video monitor A display device capable of accepting 
a video signal that is not modulated for broadcast 



either on cable or over the air. In videotapine a t»u 
vision screen on which the footage can be viewed « t 
is being recorded. 45,1 

video scan converter A device that emits a video 
signal in one standard into another device of difWnt 
resolution or scan rate. "uiercni 

video segment A contiguous set of recorded data 
from a video track. A video segment may or rnavnr* 
be associated with an audio segment 

videotape (1) A tape used to record visual imam 
and sound. (2) In multimedia applications^ 
recording of visual images and sound on nu*netic 
tape. All shooting is done in this format, even if nV 
results are later transferred to videodisc or filrrT 
(3) To make a videotape. 

videotape formats See one-inch videotape three 
quarter-inch videotape, half-inch videotape ' eirtit 
millimeter (8mm) videotape. ^ 

videotape recorder (VTR) A device for recording 

and playing back videotapes. B 

video terminal paging An IMS/VS facility that 
allows the application programmer to send multiple 
screens of information to a display device. The 
screens may then be viewed by the terminal operator 
either in or out of sequence and as many times as 
desired. 

videotex (1) A service that provides interactive 
exchange of alphanumeric and graphic mformatioii* 
over common carrier facilities to the general public 
The user must have a special display terminal or 
adapted television set. Synonymous with interactive 
videography, viewdata. (T) (2) A system that pro- 
vtdes two-way interactive information services, 
including the exchange of alphanumeric and graphic 
information, over common carrier facilities to a mass 
consumer market using ' modified TV displays with 
special decoders and modems. See also teletext. 

Note: Several versions of videotex have been devel- 
oped outside the US; for example, in Canada Telidon, 
in Germany Bildschirmtext, in Japan Captain, and in 
the U.K. Prestel. 

view (I) In an information resource dictionary, the 
combination of a variation name and revision number 
that is used as a component of an access name or of a 
descriptive name. (A) (2) In SQL, an alternative 
representation of data from one or more tables. A 
view can include all or some of the columns contained 
in the table or tables on which it is defined. (3) In 
the OS/2 operating system, the appearance of the con- 
tents of an open object. (4) In IBM network man- 
agement products, a graphical representation of a 
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ABSTRACT 

A head-mounted, wide-angle, stereoscopic display system controlled by operator position, voice 
and gesture has been developed for use as a multipurpose interface environment. The system pro- 
vides a multisensory, interactive display environment in which a user can virtually explore a 360- 
degree synthesised or remotely sensed environment and can viscerally interact with its com- 
ponents. Primary applications of the system are in telerobotics, management of large-scale 
integrated information systems, and human factors research. System configuration, application 
scenarios, and research directions are described. , 

RESEARCH OBJECTIVES 

Recent changes in requirements for human interaction with increasingly autonomous and semi- 
autonomous intelligent systems are exemplified in cockpit automation of high-performance aircraft 
and in development of automation and robotics technology for Space Station (l,2j. Although the 
long term goal of the automated space station and associated telerobotic systems is virtually auto- 
nomous operation, it is generally recognised that human operators will continue to monitor and 
supervise the systems under normal operations and to intervene under degraded modes, while also 
monitoring other space station and mission related tasks. And, although the astronaut crew will 
be less in number, less specialised and from more diverse backgrounds, their attentions will be 
divided among many disparate tasks; The projected number of workstations in the Space Station 
task environment is near 40, with workstation panels numbering over 200, and control-display ele- 
ments at 3,000 |3|. Performance in such task environments will require major functional advances 
in operator interface configurations and performance capabilities. 

The primary objective of this research is to develop a multipurpose, multimodal operator interface 
to facilitate natural interaction with complex operational tasks and to augment operator situa- 
tional awareness of large-scale autonomous and semi-autonomous integrated systems. The system 
should specifically provide a uniform interface for multiple task supervision, be reconfigurable for 
varying levels of operator skill, training and preference, and have an operator interface configura- 
tion that features human matched displays and controls for transparent, natural system interac- 
tion and reduced training requirements. 

In the Aerospace Human Factors Research Division of NASA's Ames Research Center, an interac- 
tive virtual environment display system controlled by operator position, voice and gesture has 
been developed to aid in generation and evaluation of advanced data display and management 
concepts for operator interface design. As a low-cost, multipurpose simulation device, this vari- 
able interface configuration allows an operator to virtually explore a 360-degree synthesised or 
remotely sensed environment and viscerally interact with its components. Application areas of 
the virtual interface environment research are focused on: development of workstations for telero- 
botics and telepresence control; workstations for supervision and management of large-scale 
integrated information systems; and human factors research. 
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SYSTEM CONFIGURATION 

The current virtual environment display system consists of: a wide-angle stereoscopic display unit, 
glove-like devices for multiple degree-of-freedom tactile input, connected speech recognition tech- 
nology, SD sound cueing and speech-synthesis technology, gesture tracking devices, and computer 
graphic and video image generation equipment. 

Displays 

The prototype display unit is helmet-mounted and consists of medium resolution, monochromatic 
liquid crystal display screens presented to each eye of the user through wide-angle optics (see 
Fig.l). The effective field of view for each eye is 120 degrees for horisontal and vertical with a 
common binocular field of up to 90 degrees. The prototype LCD screens measure 2. 7- inches diago- 
nally and do not yet fill the field of view completely (see Fig.2). Near term developments will 
replace these displays with 4-inch diagonal, high resolution LCD screens. For stereoscopic image 
presentation, binocular parallax cues are derived from horisontally disparate viewpoints of a 
three-dimensional computer image database or from a pair of horisontally disparate remote video 
cameras. Calibration for differences in interocular distance of different users is controlled by elec- 
tronically shifting the displayed images to the correct separation. 




Fig.l Protoypc virtual environment display system. Principal investigators: M.McGreevy k 
J.Humphries, NASA/Ames, 1985. 

As the user changes viewpoint of the displayed scene, head motion is tracked in real time by a 
helmet-mounted, 6 degree-of-freedom tracking device. The derived position and orientation data is 
used to update the displayed stereo images in coordination with the users activity. The resulting 
imagery contains full motion parallax and motion perspective information in response to the users 
movement through the virtual environment. The imagery appears to completely surround the user 
in 5-space and enables the operator to explore virtual objects and environments in real time and 
from multiple viewpoints (see Fig. 5). 

The liquid crystal display technology is used to meet design requirements for size, weight, power, 
safety and low cost (see Fig. 4). Imagery displayed on the screens is a NTSC standard video sig- 
nal generated by computer, remote video cameras, or a combination of these input media with 
other video sources such as optical video disk. Both commercial and custom video mixing and 
switching equipment is used to overlay the different image sources on a common background to 
provide multiple, interchangeable windows of information to the operator. 
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Comparison of Visual Fields 




Fig.2 Comparison of binocular field of view in the human visual system, the effective field of view 
of the virtual environment display system optics, and the field of view as limited by LCD screen 
size. ^ 

Fig. 4 Transmissive liquid crystal display screen used in the virtual environment display system. 




Fig. 3 Imagery displayed in the virtual environment appears to completely surround the user in 
3-space and enables the operator to explore virtual objects and environments in real time and 
from multiple viewpoints. 
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Aa alternative head-mounted display unit has also been developed with identical optical and 
display capabilities to the original helmet prototype (see Fig. 5). This configuration is smaller 
and lighter in weight and is designed for easier replication. The new display package is also 
implemented in a movable arm* mounted, workstation configuration for evaluation in design and 
engineering applications (see Fig. 6). 




Fig. 5 Light-weight, head-mounted display unit with wide-angle optics and LCD displays. 

In addition to the visual display, sound display technologies are used in thi4 system to provide 
additional task, system status and navigation information to the operator and to enhance overall 
situational awareness. Commercially available speech-synthesis technology with unlimited voca- 
bulary capability is used primarily for voice report requirements and acknowledgement of system 
input. Also, three-dimensional sound technology is used to provide spatially correspondent sound 
cues for objects or events both within or outside of the operator's immediate field of view. 
Delivered through headphones, the prototype 3D sound equipment generates sound cues that are 
perceived outside of the head and localised at some discrete direction and distance in 3-space 
around the user. By updating the position information of the samples in real time, the localized 
sound cues maintain their spatial positions as the user interacts with the virtual environment. 

Display Interaction 

Visceral interaction with the virtual environment surrounding the user is through speech and ges- 
ture input technologies. The system includes commercially available connected speech-recognition 
technology that allows the user to give system commands in a natural, conversational format in 
contrast to highly constrained discrete word recognition systems or keyboard input. Typical 
speech mediated interactions are requests for display/report of system status, instructions for 
supervisory control tasks, and verbal commands to change interface mode or configuration. 

For tactile interaction with the displayed three dimensional environment, the user wears light- 
weight glove-like devices that transmit data-records of arm, hand and finger shape and position to 
a host computer. The gloves are instrumented with flex-sensing devices at each finger joint, 
between fingers and across the palm of the hand (see Fig.7). Motion tracking sensors like that 
described for tracking head motion are mounted on each glove to transmit position and orienta- 
tion of the hands and arms to the host system (see Fig. 8). One application of this technology is 
to provide a three-dimensional cursor in the displayed environment. And, in coordination with 
connected speech recognition technology, the hand and arm gesture information is used to effect 
indicated gestures in the synthesised or remote environment (e.g. control of robotic arms and 




Fis.7 Tactile input glove illustrating flex-sensors, interface chip and position/orientation sensor. 
Fig. 8 Tactile input glove with position and orientation tracker. 



FISHER,et al 
NASA-Ames 



ACM Interactive 3D Graphics 
October, 1980 



end-effectors, and associated control of auxiliary camera positions). Current implementations of 
this research include a three-dimensional graphic database of an articulated hand that, in the 
display environment, is spatially correspondent with the viewer's real hand and is directly con- 
trolled by the instrumented glove device (see Fig. 9). With this capability, the operator can 
pick-up and manipulate virtual objects that appear in the surrounding virtual environment. Simi- 
larly, in the virtual data management environment, multiple windows of information and simu- 
lated control panels are positioned, tised and activated by manipulating the virtual objects in S- 
space (see Figs. 10 and 12). 




Fig. 9 SD graphic virtual objects and articulated hand directly controlled by DataGlove.; 




Fig. 10 Speech and gesture interaction in the virtual environment display system. 
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VIRTUAL INTERFACE APPLICATIONS 

Supporting Rc$tarch 

Over the past two decades, results of various specialised research programs on advanced concepts 
for human-machine interaction have gradually made possible the development of flexible, mul- 
tisensory, interactive display and control environments that can be applied to current operator 
interface design requirements. 

In the mid-WO's, scientists at Bell Laboratories developed a reconfigurable keyboard console in 
which computer generated labels are optically superimposed onto a two-dimensional array of 
pushbuttons and rapidly changed as the task requires (4). At MIT, this concept was later 
extended to a limited three-dimensional virtual workspace in which a user interactively manipu- 
lates 3D graphical objects that are spatially correspondent with hand position |S). A key concept 
in each project is the use of interactive computer graphics to create a virtual workstation environ- 
ment. 

During the same time period, researchers at MIT developed a prototype room-sized human- 
machine interface environment with wall-sized display and octophonic sound surrounding the user 
[6j. Rather than keyboard input, interaction is through speech, gesture and touch sensitive 
displays [7]. Emphasis is on creating a highly visual and personalized interface environment in 
which information access and manipulation is facilitated by discrete spatial organization and 
natural interaction [8j. 

Specialised requirements for* display environments that are closely matched to human vision capa- 
bilities have resulted in the development of head-mounted and head-coupled display concepts. Ini- 
tial research at Philco developed a head-mounted CRT for viewing a remote television camera 
coupled to operator head motion (9). Later work at the University of Utah developed a binocular 
head-mounted display that superimposes computer generated virtual objects into the real environ- 
ment of the user (10|. Numerous research efforts continue to explore the perceptual and techno- 
logical requirements for simulation of presence in remote or synthesized environments 
ill|.|12UlS|. 

The virtual environment display system developed at NASA Ames combines many of the con- 
cepts and performance capabilities of the human-machine interface developments described above 
in one integrated configuration. Following are descriptions of specific task environments for which 
this interface will be evaluated. 

Ttlerobotics 

Control of autonomous and semi-autonomous telerobotic devices and vehicles requires an interface 
configuration that allows variable modes of operator interaction ranging from high-level, super- 
visory control of multiple independent systems to highly interactive, kinaesthetic coupling 
between operator and remote system. An appropriate interface for supervisory control modes will 
provide the operator with multiple viewpoints of the remote task environment in a multi-modal 
display format that can be easily distributed and reconfigured according to changing task priori- 
ties. For remote operations that cannot be performed autonomously, the interface will need capa- 
bility to quickly switch to interactive control. In this telepresence mode, the operator is provided 
with a sufHcient quantity and quality of sensory feedback to approximate actual presence at the 
remote task site. 



r 



\ 

\ 



FISHER,et al ACM Interactive 3D Graphics 

NASA- Ames October, lose 




\ / 

Fig. 11 Virtual interface environment for telerobot control. 

The virtual environment display system is currently used to interact with a simulated telerobouc 
task environment. The system operator can call up multiple images of the remote task environ- 
ment that represent viewpoints from free-flying or telerobot-mounted camera platforms. Three* 
dimensional sound cues give distance and direction information for proximate objects and events. 
Switching to telepresence control mode, the operator's wide-angle, stereoscopic display is directly 
linked to the telerobot 3D camera system for precise viewpoint control. Using the tactile input 
glove technology and speech commands, the operator directly controls the robot arm and dex- 
terous end effector which appear to be spatially correspondent with his own arm (see Figs.9 and 

id. 

Current experimental research includes system calibration for orthoscopic image display, evalua- 
tion of operator performance in teleoperation placement tasks, and analysis of perceived localisa- 
tion of synthesised sound sources in 3- space. 

Information Management 

Similar to interface requirements for supervisory control of telerobotic systems, the operator inter- 
face for large-scale integrated information systems such as Space Station will also require a range 
of interaction modalities for operator intervention in the case of system degradation or conflicts in 
resource allocation. Efficient supervision of these automated systems will depend on highly 
graphic, multi-dimensional status representations of the numerous sub-systems in a format that 
can be easily monitored in parallel with other mission related tasks such as planning and schedul- 
ing, telerobot supervision, and communication activities. In the event of system conflict or mal- 
function, the interface environment should enable natural interaction with muti- modal, 
knowledge-based warning and advisory systems. 

Advanced data display and management concepts for this task environment are being developed 
with the virtual environment display technology. Current investigations include use of the system 
to create a display environment in which data manipulation and system monitoring tasks are 
organised in virtual display space around the operator (see Fig. 5). Through speech and gesture 
interaction with the virtual display, the operator can rapidly call up or delete information win- 
dows and reposition them in 3-space. Three-dimensional sound cues and speech-synthesis 
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technologies are used to enhance the operators overall situational awareness of the virtual data 
environment. The system also has the capability to display reconfigurable, virtual control panels 
that respond to glove-like tactile input devices worn by the operator. 

Major research issues include development of multi-modal data access and manipulation guide- 
to***'* concepts for multi-dimensional, graphic representation of knowledge-based information sys- 
}2J& *nd definition of interface configurations for shared workspace environments in collabofativT 
systems management. 




Fig. 12 Virtual interface environment for management of large-scale integrated information sys- 
tems. 

Human Factor* Research 

An additional research objective includes use of this display system as a low-cost, three- 
dimensional simulation facility to synthesise interactive test environments for aerospace human 
factors research in such areas as: spatial habitability research, rapid prototyping of display and 
workstation configurations, research on effective transfer of spatial information, and spatial cogni- 
tion research on multisensory integration. 

DISCUSSION 

Unlike most technology for 360-degree visual simulation environments, the virtual environment 
display system does not make use of large, expensive, special purpose projection configurations. 
The described system is portable and low-cost without large space and equipment requirements. 
In comparison to other research efforts in head-mounted displays, this system is unique in present- 
ing a stereoscopic image that closely matches human binocular vision capabilities and in its confi- 
guration with state-of-the-art speech and tactile input technology. 

The systems' capabilities for providing a highly graphical, uniform interface for varying task 
environments and level of interaction is critical to reduce operator workload and training require- 
ments and to increase productivity. Use of muti-modal input channels and augmentation of 
visual displays provides unobtrusive redundancy for increased accuracy and efficiency in human- 
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machine interaction. Also, multiple viewpoint presentation of task related information in a 360- 
degree, dynamic stereoscopic display will increase situational awareness and effectiveness in moni- 
toring cognitively demanding spatial tasks such as construction and proximity operations or spa- 
tial representations of knowledge-based systems and activities. 



As a research tool, the virtual environment display system follows many research efforts to 
develop operator control stations for teleoperation and telepresence but has a unique configuration 
to investigate natural, multisensory interaction with complex operational tasks. As an interface 
for data management tasks, this system is a continuation of recent research in multimodal, 
natural input technology and concentrates on development of a true three-dimensional, graphical 
interface environment that can be easily reconfigured for idiosyncratic requirements; In addition 
to specific aerospace and industrial applications discussed above, the virtual environment display 
system may find further application in such areas as education, architectural and engineering 
design simulation, medical imaging and microsurgical procedures, or entertainment and recrea- 
tion. 
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